Information theoretic approach to ground-state phase transitions for two and 
three-dimensional frustrated spin systems 
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The information theoretic observables entropy (a measure of disorder), excess entropy (a measure 
of complexity) and muhi information are used to analyze ground-state spin configurations for dis- 
ordered and frustrated model systems in 2D and 3-D. For both model systems, ground-state spin 
configurations can be obtained in polynomial time via exact combinatorial optimization algorithms, 
which allowed us to study large systems with high numerical accuracy. Both model systems exhibit 
a continuous transition from an ordered to a disordered ground state as a model parameter is varied. 
By using the above information theoretic observables it is possible to detect changes in the spatial 
structure of the ground states as the critical point is approached. It is further possible to quantify 
the scaling behavior of the information theoretic observables in the vicinity of the critical point. 
For both model systems considered, the estimates of critical properties for the ground-state phase 
transitions are in good agreement with existing results reported in the literature. 
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I. INTRODUCTION 

The standard analysis of physical phase transitions in- 
volves the analysis of orderparameters and other deriva- 
tives of the free energy An alternative approach 
is based not on physical but information-theoretic ap- 
proaches, an approach which has recently started to be 
occasionally used in the analysis of (more or less) com- 
plex systems 0. 

The presented study extends on previous studies that 
employed information-theoretic methods to measure en- 
tropy (i.e. disorder and randomness) and statistical com- 
plexity (i.e. structure, patterns and correlations) for d > 
1-dimensional systems [^-@. For ID systems, the ex- 
cess entropy constitutes a well understood information- 
theoretic measure of complexity. Effectively, it accounts 
for the rapidity of entropy convergence. While the exten- 
sion of entropy to higher dimensions is rather intuitive, 
the extension of excess entropy is not. As a remedy, in 
Ref . @ , three different approaches were developed in or- 
der to extend the definition of excess entropy to d > 1, 
allowing to quantify the complexity for spatial systems 
in higher dimensions. 

Most previous studies focused on characterizing the 
above information-theoretic observables for systems 
without disorder, only. As regards this, the local states 
method f?! , proposed for the calculation of free energies 
within importance-sampling Monte Carlo (MC) simula- 
tions, was based on entropy estimation techniques for 
lattice models with discrete interactions and translation- 
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invariant interactions (i.e. non-disordered, pure systems). 
It combined upper and lower bounds for the entropy den- 
sity to compute free energies (along with an error esti- 
mate). A comparison of numerical simulations to exact 
results for the 2D Ising ferromagnet indicated that it 
yields reliable estimates already for short simulation runs 
(even in the critical region). 

Using quite similar entropy estimation techniques, the 
simulations reported in Ref. [5| were performed for the 
2D square lattice Ising model with nearest and next near- 
est neighbor interactions. By means of single spin-flip 
Metropolis dynamics at a comparatively low tempera- 
ture, two variants of the excess entropy were put un- 
der scrutiny. A careful analysis indicated that these are 
sensitive to changes in the spatial structure of the spin 
configurations as the nearest neighbor coupling strength 
was varied. They were further found to be superior to 
conventional structure factors. This study allows to con- 
clude that the excess entropy in 2D comprises a general 
purpose measures of 2D structure. 

Only recently, Ref. [8] used similar methods to charac- 
terize local, i.e. lattice site dependent, entropies and local 
excess entropies for the Kaya-Berker model. The latter 
is based on the Ising antiferromagnet (lAFM) on a trian- 
gular lattice, wherein a particular sublattice is diluted, 
only. The lAFM exhibits geometric frustration and does 
not order at finite temperature. In contrast to the pure 
model, the Kaya-Berker model orders at finite temper- 
ature if at least a fraction p — 0.0975 of the sites on 
the diluted sublattice are deleted. The simulations were 
performed using single spin-flip Metropolis dynamics at 
fixed dilution p = 0.15, where the freezing temperature 
amounts to ~ 0.84. In the simulation, various temper- 
atures down to r = 0.4 were considered. It was found 
that the distribution of local entropies broadens in the 
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FIG. 1: GS samples for (a) the L = 64 2L> RBIM with mixed 
boundary conditions (periodic — solid line, free = dashed line) 
at = 1.15, 0.97, 0.40 (from left to right), and (b) the L = 48 
3D RFIM with fully periodic boundary conditions at 6 = 
2.0, 2.4, 2.9 (from left to right; only the spins on the x,y,z — 
planes are displayed). 



glassy phase below T w 0.8, indicating that for low tem- 
peratures local entropy is not homogeneously distributed 
over the lattice. Further, the average of the local excess 
entropy was observed to exhibit a pronounced peak at 
the critical temperature, indicating that it is sensitive to 
structural changes for the 2D configurations as a result of 
the spin glass ordering. Finally, complexity-entropy dia- 
grams for the frustrated Kaya-Berker model, recorded at 
various temperatures, were found to be qualitatively dif- 
ferent from those corresponding to pure models (see Ref. 
0). This study allows to conclude that local entropy 
density and local excess entropy are valuable observables 
that yield insight to local structure and randomness for 
frustrated 2D systems. 

Here, we aim to characterize the spatial structure dis- 
played by exact ground states (GSs) of disordered model 
systems in terms of the information-theoretic observables 
entropy and excess entropy. For this purpose we consider 
the 2D random bond Ising model {2D RBIM) as well as 
the 3D random field Ising model {3D RFIM) , which both 
exhibit disorder-driven zero-temperature phase transi- 
tions. More precise, we use one of the approaches devel- 
oped in Ref. [5] , aimed at understanding spatial patterns 
for 2D systems by parsing them into ID sequences. In 
addition we also consider the multi-information, which, 
for the ordinary Ising ferromagnet in the thermodynamic 
limit was proved to be maximized at the critical point 
0. In contrast to the previous studies we work at T = 0, 
aiming to characterize the ground-state phase transitions 
that appear as the disorder is varied for the above two dis- 
ordered model systems (see sect.|lT]). In doing so, we were 
interested whether one can observe a change in the struc- 
ture of the ground states by using the above information- 
theory inspired observables. Further, if the latter is pos- 
sible, it is of interest to quantify the scaling behavior of 



these observables in the vicinity of the phase transition. 

The remainder of the presented article is organized as 
follows. In section [n] we introduce the spin models that 
arc considered in the presented study. In section Hill the 
information-theoretic observables entropy and excess en- 
tropy are introduced and illustrated in more detail. Sec- 
tion llVl reports on the numerical results, further discussed 
in section |Vl An elaborate summary of the presented ar- 
ticle is available at the papercore database jlO| . 

II. MODELS 

Next, we present the two models which were studied 
in this work, the 2D RBIM and the 3D RFIM. 

A. 2D random bond Ising model 

We investigated GSs for the 2D RBIM, considering a 
square lattice of side length L. The respective model 
consists oi N = Ising spins, for which a particular 
spin configuration might be written as it = (ai, . . . , ctat), 
where ai € {-1-1,— 1}. The energy of a given spin config- 
uration is measured by the Edwards- Anderson Hamilto- 
nian 

^^RBIm(o') = — ^ Jij CTiCj, (1) 

ihj) 

where the sum is understood to run over all pairs of 
nearest- neighbor spins (on a 2D square lattice), with pe- 
riodic boundary conditions (BCs) in the x-direction and 
free BCs in the y-direction. In the above energy function, 
the bonds Jij are quenched random variables drawn from 
the disorder distribution 

P(J„-) = exp[-(J,, - /i)V(2a})]/(ajV2^), (2) 

where the width of the distribution was fixed to ctj = 1. 
Consequently, one realization of the disorder consists of 
a mixture of antiferromagnetic bonds {Jij < 0) that pre- 
fer an antiparallel alignment of the coupled spins, and 
ferromagnetic bonds {Jij > 0) in favor of parallel aligned 
spins. In general, the competitive nature of these in- 
teractions gives rise to frustration. A plaquette, i.e. an 
elementary square on the lattice, is said to be frustrated 
if it is bordered by an odd number of antiferromagnetic 
bonds. In effect, frustration rules out a GS in which all 
the bonds are satisfied. As limiting cases one can identify 
the random bond Ising ferromagnet (FM) as /i — )■ oo and 
the (Gaussian) 2D Edwards Anderson Ising spin glass 
(SG) [TT'JTs'] at /X = 0. A GS spin configuration acs is 
simply a minimizer of the energy function Eq. [1] Thus, 
regarding the GSs as a function of the variable /i, we ex- 
pect to find a ferromagnetic phase (spin glass phase) for 
fi > fic {iJ- < fJ-c) wherein fj,c denotes the critical point 
at which the T = FM-SG transition, i.e. a continuous 
disorder-driven phase transition, takes place. Samples of 
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GSs for the 2D RBIM for different values of the param- 
eter /i are shown m Fig. ma). 

The Ising spin glass is a paradigmatic model for a dis- 
ordered magnet. Since the effects of the disorder are well 
visible at zero temperature, the investigation of ground 
state properties is of prime importance. For a planar ver- 
sion of this model, e.g., a 2D square lattice with periodic 
boundary conditions in only one direction, a solution of 
the GS problem is possible by means of a mapping to 
an appropriate minimum-weight perfect-matching prob- 
lem. This latter problem can be solved by means of 
exact combinatorial optimization algorithms from com- 
puter science [12, 14 18], whose running time increases 
only polynomially with the system size. Hence, very large 
systems can be treated exactly, giving very precise and 
reliable estimates for the observables. The GS problem 
for lattice dimensions d > 2 or systems subject to an ex- 
ternal magnetic field belong to the class of nondetermin- 
istic polynomial (NP)-hard problems For those 

problems, no exact algorithm with a polynomial running 
time has been found so far. From a conceptual point of 
view, the existence of numerically exact and highly effi- 
cient algorithms for the 2D SG with periodic boundary 
conditions in at most one direction motivates the special 
interest in this setup during the last decades. 

As pointed out above, for the 2D RBIM on planar lat- 
tice graphs (including the Ising spin glass), GS spin con- 
figurations can be found in polynomial time. We here use 
a particular mapping to an appropriate minimum- weight 
perfect matching problem, presented in Ref. [ll] . The use 
of this approach permits the treatment of large systems, 
easily up to L = 512, on single processor systems. In a 
previous study [20| . we performed such GS calculations 
and employed a finite-size scaling analysis for systems 
of moderate sizes [L < 64) to locate the critical point 
at which the transitions takes place. Therefore, we ana- 
lyzed the Binder parameter [21| 6l = |[3— (m|^)/(m|)^] 
that is associated to the magnetization and is ex- 
pected to scale as 6l(p) ^ f[{p — Pc)L^^^], where / is a 
size-independent function and v signifies the critical ex- 
ponent that describes the divergence of the correlation 
length as the critical point is approached. The magneti- 
zation is simply the sum of all spin- values in the GS spin 
configuration, i.e. rriL — J2i'^GSi/L'^- Using the data 
collapse anticipated by the scaling assumption above we 
obtained = 1.031(2) and i^'" = 1.49(4) (for a fur- 
ther study of this transition, yielding similar results using 
renormalization group techniques, see Ref. (22j'). Finally, 
we characterized the transition using a finite-size scaling 
analysis for the largest and second-largest ferromagnetic 
clusters of spins within the GS spin configurations. These 
clusters form as one proceeds from spin glass ordered to 
ferromagnetic ground states. The results obtained from 
the related scaling analysis support the results obtained 
earlier by considering the Binder parameter and magne- 
tization. 



B. 3D random field Ising model 

We further investigated GSs for the 3D RFIM on a sim- 
ple cubic lattice of side length L. The respective model 
consists of = Ising spins, and the energy of a given 
spin configuration is measured by the Hamiltonian 

where the first sum is understood to run over all pairs 
of nearest-neighbor spins on a 3D simple cubic lattice 
with fully periodic BCs. In the above energy function, 
the bonds that couple adjacent spins are ferromagnetic, 
i.e. J > 0, and the local fields bi (i = 1, . . . , N) introduce 
disorder to the model. The values of the local fields are 
independently drawn from the disorder distribution 

P{h)^exp[^{hr/{2b^)]/{bV2^), (4) 

where the mean of the distribution is fixed to zero and the 
width amounts to b. Thus, one realization of the disorder 
consists of ferromagnetic spin-spin couplings with each 
spin coupled to a local random field. 

The RFIM is a basic model for random systems [l^, 
[23 | and also gives rise to frustration. While in order 
to minimize Eq. [3] the ferromagnetic spin-spin coupling 
will tend to align coupled spins in parallel, the random 
fields will tend to align the spins parallel to the local 
field, possibly introducing a paramagnetic effect on the 
GSs. For a field width that is small compared to the 
ferromagnetic coupling, i.e. for b J, one might expect 
a dominance of the ferromagnetic spin-spin coupling in 
the GS spin configurations. In contrast to this, for a 
comparatively large field width, the orientation of the 
majority of spins in a GS will be determined by the local 
random fields, suggesting a paramagnetic GS for large 
values of b. As a result, at T = 0, the RFIM exhibits a 
disorder driven, continuous ferromagnet to paramagnet 
(PM) transition regarding the ground state structure at 
a finite value 6c. Samples of GSs for the 3D RFIM for 
different values of the field strength b are shown in Fig. 

The solution of the GS problem for the general RFIM 
(not only its 3D variant) is possible by means of a map- 
ping to an appropriate maximum-fiow problem [T^ . [25l . 
|26|. This latter problem can be solved in polynomial 
time by means of exact combinatorial optimization algo- 
rithms from computer science. As for the 2D RBIM this 
offers the possibility to study large systems, easily up to 
i = 64, giving very precise and reliable estimates for the 
observables. 

Using such optimization methods, the critical point 
and critical exponents for the GS phase transition can 
be obtained by analyzing the magnetic properties of the 
model d^-ii]. For the 3D RFIM we here quote the val- 
ues bf = 2.28(1) for the critical point and i/"* = 1.32(7) 
for the critical exponent that describes the divergence of 
the correlation length as the critical point is approached. 
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Further, the 3D RFIM also exhibits a particular perco- 
lation transition. I.e., regarding the simultaneous span- 
ning of up and down spin domains as a function of the 
field strength b, Ref. 30] report on the percolation criti- 
cal point = 2.32(1) and the value z/^" = 1.00(5) (note 
that for random percolation in 3D one has Vp — 0.88; 
see Ref. [3l|). However, the results appeared to depend 
on the particular spanning criterion. In this regard, in 
the limit of the spanning probability approaching to zero 
(indicating the onset of percolation) the scaling analysis 
extrapolated to Vp*" — 1.3(1). 



III. INFORMATION-THEORETIC 
OBSERVABLES IN ID AND EXTENSION TO 

d = 2 AND 3 

In the current section we introduce basic notations 
from information theory, needed to define the entropy 
rate, excess entropy, and multi information that might 
be associated to a (ID) sequence of symbols. In this re- 
gard, for the definition of the entropy rate and excess 
entropy we follow the notation of Refs. @, IH, [s^. For 
the definition of the multi information we follow Ref. Q . 
In the above references, a more elaborate discussion of 
the individual information-theoretic observables can be 
found. 

A prerequisite for the definition of these observables 
is the M-block Shannon entropy H[S'^^] for a block of 
M consecutive random variables S^^ = Si . . . Sm- Each 
random variable Si might assume values Ui £ A, where 
the set A denotes a finite alphabet (subsequently, the 
random variables will be identified with Ising spins, in 
which case A will denote the binary alphabet {— 1, +1}). 
For = (Ti . . . aM denoting a particular symbol block 
of length M > 0, the Af -block Shannon entropy reads 

H[S^]^^ J2 Pr(a*0log2(Pr(a*^)), (5) 

where Pr(cr^^) signifies the joint probability for blocks 
of M consecutive symbols (i.e. spin orientations in the 
present context). In the above formula, the sum runs over 
all possible blocks, i.e., combinations of of M consecutive 
symbols from A. 



A. Entropy density, excess entropy, and multi 
information for one-dimensional symbol sequences 

Using the A/-block Shannon entropy, the asymptotic 
entropy density for a ID system is given by 

lim H\S^^]/M. (6) 

M^QC 

A sequence of finite-M approximations h{M) to the 
asymptotic entropy density that typically converges 



faster than the expression above is provided by the con- 
ditional entropies 

h{M) = H[Sm\S^^'^] - H[S^^] - H[S^^-\ (7) 

As given above, h{M) denotes the entropy of a single 
variable (spin) conditioned on a block of M — 1 adjacent 
variables (spins). Thus, h is the randomness that still 
remains, even after correlations over blocks of infinite 
length are accounted for. Viewed as a function of block 
size, the finite-M conditional entropies converge to the 
asymptotic value h from above. Hence, at small length 
scales the system tends to look more random than it is 
in the limit M — >■ oo. 

For one-dimensional systems, there are three different 
but equivalent expressions for the excess entropy. These 
are based on the convergence properties of the entropy 
density, the subextensive part of the block entropy in the 
limit of large block sizes, and, the mutual information 
between two semi-infinite blocks of variables, see Refs. 
[^Q. Here, we focus on the definition of excess entropy E 
that relates to the convergence properties of the entropy 
density in the form 

oc 

E = Y,[h{M) - h]. (8) 

M=l 

The conditional entropies h{M) constitute upper bounds 
on the entropy rate, allowing for an improving estimate of 
h for increasing M . Thus, the individual terms in the sum 
comprise the entropy density overestimates on the level 
of blocks of finite length M . In total, the excess entropy 
measures the area between h{m) and the horizontal line 
at /i. As such, E accounts for the randomness that is 
present at small lengths and that vanishes in the limit of 
large block sizes. 

In \D and in the limit of large block sizes, the multi 
information is given by the first summand in Eq. |51 i.e. 

/ = h{l) - h. (9) 

Albeit / is closely related to E (this holds only in the limit 
of large block sizes M; see Ref. [H for a more general dis- 
cussion of the multi information), it captures somewhat 
different characteristics regarding the convergence of the 
entropy density. In this regard it measures the decrease 
of average uncertainty in the description of the system by 
switching from the level of single variables (spins) statis- 
tics to the statistics attained as M — >■ oo. 



B. Extension to d = 2 and 3 

Following Ref. ^ , the most general approach designed 
to write the 2D entropy density as the entropy of a partic- 
ular variable conditioned on a block of neighboring vari- 
ables (similar to the II? case), uses so called neighbor- 
hood templates of a given size. In general, for a neighbor- 
hood template only spins in the same row left of a "target 
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FIG. 2: Neighborhood templates for the definition of the con- 
titional entropies (see Eq. [T]) for (a) the 2D RBIM, and (b) 
the 3-D RFIM. The target variable is represented by a shaded 
cell and the numbers within the cells reflect the order in which 
the sites are picked in order to compute the condititional en- 
tropies (see text). 



spin" and the spins in all rows below the target spin are 
considered. Fig. [If a) shows such a finite-size template 
for a 2D system, covering an overall number of 25 spins. 
Similar to Ref. [5] we use the parameter M in the condi- 
tional entropy (Eq. [7]), to account for a successive addi- 
tion of single sites from the neighborhood template shown 
in Fig. [Jfa). Therein, the numbers within the cells indi- 
cate the order in which the lattice sites are added to the 
2D neighborhood template (from initial numerical exper- 
iments we found that it is not necessary to exceed M = 10 
(M = 6) for the 2D RBIM {2,D RFIM)). In this regard, 
let the subscript {/S.x, Ay) denote the relative position of 
a variable S in the neighborhood template, specifying its 
distance to the target variable in terms of site-to-site hops 
(in particular the target variable is labeled ^'(o.o))- Then, 
the order in which the sites are picked from the template 
reflects the increasing Euclidean distance of the particu- 
lar lattice site to the target site. Thereby, we follow the 
convention that if there is a draw regarding the distance 
of two or more lattice sites, we add them to the neigh- 
borhood template from the top left to the bottom right, 
increasing the value of Ay before Ax (see the order of 
spins 7 through 10 in Fig.t^Ia)). As an example note that 
h{6) = 7J[S'(o^o)l'5'(o, -i)'S'(i,o)'5'(i,-i)S'(i, 1)5(0 _2)]- We fol- 
lowed a similar approach in SD, where the corresponding 
neighborhood template is shown in Fig. [5Jb), and where 
e.g. h{A) = i?[5'(o,o,o)|5'(o,o,-i)5'(o,i,o)5'(-i,o,o)]- 

Note that by the procedure discussed above, we disre- 
garded the practice that if the interactions between the 
variables are of finite range, the neighborhood template 
only needs to be as "thick" as the interaction range d, 0] . 
For disordered model systems that exhibit competing in- 
teractions, nontrivial correlations between the spin de- 
grees of freedom might emerge that extend towards the 
intrinsic range of the interactions (for a brief discussion 
of this issue and its implications, in particular for the 
design of an efficient cluster algorithm for the fully frus- 
trated Ising model, see Ref. [SJ])- Consequently, for the 
2D RBIM and 3D RFIM we considered it more adequate 
to include more than just the nearest neighbors for the 
construction of a neighbor template. 
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FIG. 3: Results for the entropy density of the 2D RBIM with 
side length L = 384. (a) illustrates the convergence of the 
average entropy density {h[M)) as a function of the neigh- 
borhood size M for three values of the disorder parameter ^l. 
(b) shows the average entropy density for different neighbor- 
hood sizes as a function of the disorder parameter /i. 

Subsequently, 2D and ZD configurations of spins are 
analyzed by parsing them into one dimensional sequences 
using the order of the spins as illustrated in Fig. [51 The 
resulting sequences can then be analyzed using Eqs. [7] 
and [51 above. From a practical point of view it will sub- 
sequently be necessary to truncate the sums in Eqs. [3 [H 
to a maximally feasible neighborhood size A/j^ax ■ 



IV. RESULTS 

Here, our approach is somewhat different than that 
reported in Ref. [8|. In that study, one particular real- 
ization of disorder for the Kaya-Berker model was put 
under scrutiny at a particular value of the dilution pa- 
rameter and for different temperatures (above as well as 
below the freezing temperature of the system). For that 
particular disorder instance and for a given temperature, 
a single-spin flip Metropolis dynamics was used to gener- 
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FIG. 4: Results for the 2D RBIM for different square lattices with side length up to L = 512. (a) The main plot shows the 
average excess entropy (E) as a function of the mean jj, of the bond disorder distribution (lines are guides to the eyes only). 
The inset illustrates the finite-size scaling of the peak position, the line shows the result of a fit to Eq. 1101 For all data-points 
error bars are obtained via bootstrap resampling, (b) The main plot shows the finite-size susceptibilities xe{L) = L^var(i5) 
associated to the excess entropy (lines are guides to the eyes only). The inset signifies the finite-size scaling of the peak width 
obtained from a fit of the data points close to the peak to a Gaussian function, the line shows the result of a fit to Eq. 1111 
(c),(d) are the same as (a),(b), respectively, but for the multi-information I. 



ate independent spin configurations. These spin config- 
urations were then used to accumulate statistics for spin 
blocks (specified by the utilized neighborhood template) 
that provide means to estimate conditional entropies on 
either local (i.e. lattice site dependent) or global scale, 
where the latter is a spatial average of the local condi- 
tional entropies. As pointed out by the authors of Ref. 
Q, it is important to accumulate spin-block statistics in 
a lattice site dependent manner so as to not overestimate 
the entropy of the system. A later average over the local 
observables yield the correct thermodynamic entropy, as 
verified by the authors. 

However, note that here we work at T = 0, aiming to 
characterize ground state phase transitions for disordered 
model systems in 2D and 31?. I.e. for each realization of 
the disorder (and due to the particular disorder distribu- 
tions used in the presented study) , there is one particular 



spin configuration that qualifies as a ground state (aside 
from a trivial degeneracy stemming from the global spin- 
fiip symmetry of the energy function Eqs.[TJ As a remedy, 
in order to collect statistics for blocks of spins we sweep 
the neighborhood template over the full lattice (as one 
would do for pure systems, see Ref. Q), thus averaging 
over different "local" configurations of the disorder. For 
one ground state and for a chosen block size M, this 
yields one spatially averaged estimate for the conditional 
entropy on the level of M-spin blocks. The results are 
then averaged over many realizations of the disorder. In 
doing so, we were interested whether one can observe a 
change in the structure of the ground states by using 
the above information-theory inspired observables, and 
how this compares to phase transitions, which were pre- 
viously observed when using standard statistical-physics 
observables. 
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Furthermore, it is of interest to quantify the scahng 
behavior of these observables in the vicinity of the phase 
transition. Note that we also performed additional sim- 
ulations using the definition of 2D and 3D neighbor- 
templates as used in Ref. ,7] and found no qualitative 
difference to the results reported below. 



A. Results for the 2D RBIM 

a. Entropy density: As evident from Fig. Efa) , the 
entropy density for the 2D RBIM for L = 384 exhibits 
a rapid convergence. In this regard, for a neighborhood 
size of M = 5 the average entropy density appears to 
take its asymptotic value for the full range of considered 
parameters /i. However, for the numerical simulations at 
L = 384 we considered the maximally feasible neighbor- 
hood size Afniax = 10. For smaller system sizes, M^ax 
had to be adjusted to somewhat smaller values to assure 
that for a given system size L the measurement of the 
conditional entropies at a given level h{M) are based on 
sufficient statistics for the underlying M-block shannon 
entropies. In this regard, at L = 64 we used M^ax = 6. 
The average entropy densities, restricted to neighbor- 
hood sizes M — 1 through 10, are shown in Fig. [Sjb). 
It can be seen that for small (large) values of /i the en- 
tropy density converges to a comparatively large (small) 
value representing the disordered (i.e. SG ordered) and 
ordered (i.e. ferromagnetic) phase, respectively. Espe- 
cially the curve corresponding to M = 1 exhibits a steep 
decrease in the interval /i S [1, 1.1]. Moreover, the fluc- 
tuations va.r{h{fi)) = (/i(^)^) — {h{^))^ of the entropy 
density are peaked at « 1.025 (not shown). As it ap- 
pears, this parameter value is close to the asymptotic 
critical point fj}^^ = 1.030(2) that indicates the T = SG 
to FM transition for the RBIM [13, [l!] in the limit of 
large system sizes. 

b. Excess entropy: A finite-size scaling analysis of 
the system size dependent peak position of the average 
excess entropy (E) (see Fig. IH^a)) was performed in the 
following way: polynomials of order 5 were fitted to the 
data curves at different system sizes L in order to obtain 
an estimate Hc,i{L) of the peak position. Thereby, the in- 
dex i labels independent estimates of the peak positions 
as obtained by bootstrap resampling [35j . For the analy- 
sis, we considered 40 bootstrap data sets, e.g. resulting in 
the final estimate = 256) — 0.975(1). Anticipating 
the scaling form 



(a) 



Hc{L) ^ pic - aL' 



(10) 



see inset of Fig. Ufa), and considering the fit interval 
L £ [64,512] yields the fit parameters fXc = 0.995(5), 
6 = 1.0(1) and a = 0{1) for a reduced chi-square 
xVdof = 1.19 (dof = 4). Note that as L ^ oo, 
the location of the peak disagrees with the location 
ixf = 1.031(2) of the T = SG to FM transition. 
The latter estimate was obtained from an analysis of the 
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FIG. 5: Results for the entropy density of the ZD RFIM with 
side length L — 64. (a) illustrates the convergence of the aver- 
age entropy density {h{AI)) as a function of the neighborhood 
size Ai for three values of the field strength b. (b) shows the 
average entropy density for different neighborhood sizes as a 
function of the field strength b. 



binder-parameter for the GS magnetization [20'|. Ear- 
lier simulations, using a transfer-matrix approach at fi- 
nite temperature and extrapolated to T = 0, report on 
M^"' = 1/re = 1.04(1) [12. 

A similar scaling analysis for the location of the 
peaks related to the finite-size susceptibilities xe(L) = 
L^-vai{E) by means of Gaussian fit-functions results in 
the estimate pL^ = 1.029(1). Further, the width of the 
Gaussian fit-function obeys the scaling 



u{L) = aL 



-b 



(11) 



where a = 0(1) and b = 0.65(2) (xVdof = 0.45, dof = 
5). Note that the inverse of the latter fit parameter reads 
1/6= 1.54(5) and is thus strikingly close to the critical 
exponent = 1.49(7) ;20] (i^'"* = 1.42(8) [H) that 
characterizes the T = SG to FM transition. 

c. Multi-information: A scaling analysis of the sys- 
tem size dependent peak position displayed by the av- 
erage multi- information (/) (see Fig. IHc)) according to 
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FIG. 6: Finite-size scaling analysis for the 3-D RFIM on cubic lattices with side length up to i = 64. (a) the main plot shows 
the average excess entropy (E) as a function of the field strength b. (For this and the other main plots, lines are guides to 
the eyes only). The inset illustrates the finite-size scaling of the peak position, the line, as for the other insets, shows the 
result of a fit to Eq. [121 For a-U data-points error bars are obtained via bootstrap resampling, (b) the main plot shows the 
finite-size susceptibilities Xe{L) = L^vslv{E) associated to the excess entropy and the inset indicates the finite-size scaling of 
the associated peak location. (c),(d) are the same as (a),(b), respectively, but for the multi-information /. 



the scaling assumption Eq. [TU]in the range L £ [64, 512] 
yields the fit-parameters /ic = 1.026(3), a = 0(1), 
and b = 0.63(4), i.e. 1/b = 1.6(1) (xVdof = 0.20, 
dof — 4). Neglecting the smallest system size, i.e. re- 
stricting the fit interval to L g [96,512], leads to the 
estimate fic = 1.031(6) which is even closer to the known 
critical point. 

Further, an analysis of the position and width of the 
peaks related to the finite-size fluctuations xi = i^var(/) 
by means of Gaussian fit-functions results in the addi- 
tional estimate = 1.032(5) (see Fig. HJd)). As above, 
the widths (j{L) obay the scaling form of Eq. [TTJ where 
a fit yields a = 0(1) and b = 0.65(2), i.e. 1/b = 1.54(5) 
(xVdof = 0.32, dof = 5; inset of Fig. Hd)). 

Note that here, both estimates of /ic and both esti- 
mates of the exponent 1/b are reasonably close to those 
found earlier for the 2D RBIM. Hence, this might indi- 
cate that in comparison to the full excess entropy, the 



multi-information is more sensitive to structural changes 
at the T ~ order-to-disorder transition in the 2D 
RBIM. 

d. Geometric properties of the GSs: Upon increas- 
ing the value of the disorder parameter from ii — to 
fjL > jjic, it is possible to identify ferromagnetic clusters of 
spins with increasing size. A finite size scaling analysis 
of the relative size of the largest and second largest fer- 
romagnetic clusters of spins for the independent GSs can 
be utilized to characterize the T = SG to EM transi- 
tion in the 2D RBIM. E.g. as reported in Ref. [l^, the 
relative size of the largest ferromagnetic cluster scales as 
(Afi) cx L-l^/^flip - tic)L^/% where a data collapse (for 
system sizes L — 24 . . . 64) yields the scaling parame- 
ters pL^^ = 1.032(2), i/f" = 1.49(4), and P^'' = 0.039(4). 
Note that the results obtained for the multi-information 
is also in excellent agreement with those obtained from 
an analysis of the largest cluster size, which constitutes 
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an observable that links to the geometric properties of 
the GS spin configurations. 

B. Results for the 3D RFIM 

e. Entropy density: As can be seen from Fig. [Sja), 
the entropy density for the iD RFIM (the figure shows 
the data for L — 64) convergence rapidly. I.e. for a neigh- 
borhood size of M = 6 the average entropy density ap- 
pears to take its asymptotic value for the full range of 
considered field strengths b. During the numerical simu- 
lations, carried out on cubic systems of side length L = 8 
through 64, we thus considered the maximally feasible 
neighborhood size Mmax = 6. The average entropy den- 
sities for M = 1 ... 6 are shown in Fig. ^h). For the 
3-D RFIM the ordered, i.e., ferromagnetic (FM)) and 
disordered, i.e., paramagnetic (PM) phases are located 
at small and large values of the disorder parameter &, 
respectively. As discussed above, the curve correspond- 
ing to n = 1 exhibits a steep decrease in the interval 
b G [2.2, 2.5], which already allows to shed some light on 
where the characteristics of the model at T = change 
from ferromagnetic to paramagnetic. In a previous study, 
using the finite-size scaling of standard physical quanti- 
ties, the critical strength of the random field at which the 
T = FM to PM transition takes place was found to be 
bf = 2.28(1) [29]. 

/. Excess entropy: The scaling analysis of the sys- 
tem size dependent peak position of the average excess 
entropy (E) (see Fig.lSfa)) was performed similar to the 
2D RBIM above. Assuming that the system size depen- 
dent peak locations bc{L) exhibit a scaling of the form 

b,{L)^bc~aL-^, (12) 

see inset of Fig. EKa), yields the fit parameters be = 
2.38(1), c = 0.81(2) and a = 0(1) (xVdof = 0.05, 
dof = 4). Note that as L — >■ cx), the location of the 
peak extrapolates to a value that is close by the criti- 
cal point 6p* = 2.32(1), describing the transition of the 
probability for a simultaneous spanning of up and down 
spin domains from 1 to 0, see Ref. [30]. A similar scal- 
ing analysis for the location of the peaks related to the 
finite-size susceptibilities xe{L) = L^var(£'), again by 
means of polynomials of order 5, results in the estimate 
be — 2.23(1), see inset oflHKb). As it appears, the es- 
timates of 6c obtained from the excess entropy and its 
fluctuation do not match up well. 

g. Multi-information: A finite-size scaling analysis 
of the system size dependent peak position shown by the 
average multi-information (/) (see Fig.[6Kc)) according to 
the scaling assumption Eq. [T2]in the range L E [12,64] 
yields the fit-parameters be = 2.34(1), a — 0(1), and c = 
0.79(3) (xVdof = 0.22, dof = 4). Further, an analysis of 
the peak location for the related finite-size fiuctuations 
Xi = L^vai{I) results in the estimates 6c — 2.30(1), a ~ 
0(1) and c = 0.80(7) (xV^of = 0.26, dof = 5); see Fig. 
Eld). 



Note that here, both estimates of 6c and both esti- 
mates of the exponent c match up well. Also, the nu- 
merical estimates 6c agree with the numerical value of 
6p' within errorbars and the numerical value of 1/c — 
1/0.80(7) — 1.3(1) (obtained by means of the above anal- 
yses) is in good agreement with the correlation length 
exponent = 1.3(1) reported in Ref. [2£i]. Further, 
they are in good agreement with the numerical values to 
which the scaling analysis for the percolation criterion in 
Ref. [13] extrapolates to. Again, these results indicate 
that the multi-information is very sensitive to structural 
changes in GS spin configurations at the T = order-to- 
disorder transition. 



C. The T — phase transition in purely 
information theoretic coordinates 

In the above analyses the information theoretic co- 
ordinates entropy density h and excess entropy E, giv- 
ing measures of randomness and complexity, respectively, 
were studied as a function of a model specific parame- 
ter. So as to facilitate a comparison of different models 
complexity- entropy diagrams are of great use A sur- 
vey of complexity-entropy diagrams for different model 
systems can be found in Ref. 6] • The complexity-entropy 
relationship for the 2D RBIM and 3D RFIM for different 
system sizes are shown in Fig. [T] As evident from the fig- 
ure, the curves for both models exhibit similar features. 
I.e. they have an isolated peak at a particular value of 
h. This is similar to the complexity-entropy curve for 
the 2D Ising FM, where the corresponding peak is lo- 
cated at entropy density (h) « 0.57 with a peak height 
of (E) w 0.4, see Ref. p. In order to illustrate finite-size 
effects for the complexity-entropy diagram, three differ- 
ent system sizes for both models are shown. A scaling 
analysis of the finite-size peak locations indicates that for 
the 2D RBIM {3D RFIM) the peak shifts to the entropy 
density value he = 0.26(1) (he = 0.107(1)) as L ^ oo. 
Comparing with the result of the FM mentioned before, 
one can say that for the 2D RBIM and the 3D RFIM, 
the order-disorder transition appears at smaller entropy 
but is connected with a higher complexity. Note, for the 
considered models, the complexity-entropy diagram con- 
tains the FM phase in the parameter range h G [0,hc]. 
The SG phase for the 2D RBIM (PM phase for the 3D 
RFIM) is found for he[hc,l]. 



V. DISCUSSION 

Regarding the excess entropy, conceptually similar 
analyses carried out on spin configurations for the 2D 
Ising FM, reported in Ref. @|. The respective study con- 
cluded that the excess entropy is peaked at a tempera- 
ture T w 2.42 in the paramagnetic (i.e. disordered) phase 
above the true critical temperature Tc = 2.269. Qualita- 
tively similar results on the mutual information for the 
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FIG. 7: Complexity-entropy diagram for the 2D RBIM and 
3-D RFIM for different system sizes L. Note that for the 
complexity-entropy curve for the 2D Ising FM, the corre- 
sponding peak is located at entropy density (h) ~ 0.57 with 
a peak height of (E) ^ 0.4, see Ref. 0. 

2D Ising FM (and more general classical 2D spin models) 
were recently presented in Ref. [37]. There, the authors 
conclude that the mutual information (equivalent to the 
excess entropy for ID systems; see Ref. js^l) reaches a 
peak- value in the paramagnetic phase close to the system 
parameter K = J/ksT « 0.41 (for J — ~ I this cor- 
responds to T « 2.44), again in disagreement with the 
critical point of the underlying model. Further, the au- 
thors note that the true critical point appears to coincide 
with the inflection point where the first derivative of the 
excess entropy tends to minus infinity in the thermody- 
namic limit (however, the authors present no systematic 
analysis of this observation). Hence, it is not too surpris- 
ing that a finite-size scaling analysis for the peaks of the 
bare excess entropy does not directly allow to identify 
the critical point of the model considered here. Similar 
to the previous studies for the Ising FM, we here find 
that the excess entropy assumes an isolated peak at a 
parameter-value located slightly below the true critical 
point in the disordered phase. However, a scaling analy- 
sis reveals that in the thermodynamic limit, the peak of 
the related finite-size fluctuations is located right at the 
critical point. 



The multi-information was introduced by Erb and Ay 
in Ref. where the authors considered the multi- 
information to characterize spin-configuration for the 2D 
Ising FM in the thermodynamic limit by analytic means. 
Among other things, the authors conclude that the multi- 
information exhibits an isolated global maximum right 
at the critical temperature (see Theorem 3.3 of Ref. [9|). 
Here, we find that in contrast to the excess entropy and 
in qualitative agreement with analytic results for the 2D 
Ising FM, the peak of the multi-information tends to- 
wards the critical points reported in the literature as 
i — > oo. However, the critical point to which the ef- 
fective, system size dependent critical points related to 
the peaks of the excess entropy for the 3D RFIM con- 
verges is in good agreement with the precise location 
that corresponds to a particular percolation transition 
for the respective spin model. Nevertheless, given the so 
far achieved numerical accuracy, it is still not clear to us 
whether these two transition points are really distinct. 
Anyway, both observables studied in this work basically 
turned out to be useful information-theoretic measures 
that might be used to distinguish the ordered and disor- 
dered phase of frustrated model systems, as e.g. the 2D 
RBIM and 3D RFIM. Further, the finite size scahng of 
these observables allows to estimate critical points and 
exponents that are in good agreement with the critical 
properties reported in the literature. 

Given these promising results for systems exhibiting 
quenched disorder, it would be of particular^ interest to 
apply these methods to structural glasses [38|], where a 
simple way of analyzing snapshots of configurations is 
still missing [sof . 
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